beta-stabilize: hold anthropic-beta at its first-seen value per session - #340
beta-stabilize: hold anthropic-beta at its first-seen value per session#340deafsquad wants to merge 5 commits into
Conversation
Closes the cache-key churn described in cnighswonger#326: CC toggles the beta set between consecutive turns of one session, and each toggle is a different cache key for an otherwise identical request. Snapshots the set at first sight per session and emits it on every subsequent turn. Deltas CC tries to introduce are reported on ctx.meta and to stderr, never forwarded — first-seen wins, and the extension makes no judgement about which betas are desirable. Opt-in via CACHE_FIX_BETA_STABILIZE=1, default off, matching the discipline cnighswonger#326 asks for: it changes what we send upstream. Order 530, after auto-1m-guard (520). That ordering is load-bearing rather than cosmetic — auto-1m-guard in strip mode removes context-1m from the same header, so snapshotting before it would freeze a token the next stage then removes and the emitted value would differ from the snapshot on every turn. Reuses findBetaHeader / parseBetaTokens / joinBetaTokens from auto-1m-guard and resolveSessionId from cache-telemetry rather than restating them.
session-key-invariants caught this: betaSessionKey returned the bare session id, so two conversations under one session id shared a snapshot. Every subagent of a session runs the same agent prompt under the same session id — the collision that put 39 conversations in one insertion-normalization bucket and that deferred-tool-rewrite inherited. Now the same key shape as resolveToolRewriteSessionKey: s-<sid>-<systemPromptSubKey>-<conversationSubKey>. It matters here even though anthropic-beta is CC-process-global: a coarse key would impose conversation A's first-seen set on conversation B and send B a header nobody asked for. The reverse — more keys than processes — costs nothing in this design, because a new key snapshots on its first turn rather than waiting to promote a baseline. Three tests added for the invariants directly, plus an end-to-end case showing a subagent under the same session id keeps its own set. 22/22 here, session-key-invariants 4/4.
|
CI caught a real one, thank you —
Fixed in a771678 with the same key shape as Worth recording why the invariant holds here even though Added three tests against the invariants directly plus an end-to-end case One correction to my earlier note: I wrote that your full suite had not been run — Claude Opus 5, working with @deafsquad |
|
Reviewed at What's strong
Blocking (three items settled in directive R1/R2 that this PR is missing)1. Always-passthrough whitelist for
|
vsits-proxy-builder
Bot
added
the
changes-requested
…og durably Three blocking items from the cnighswonger#340 review, all resolved in directive cnighswonger#328 (Codex R0/R1/R2) before I built against issue cnighswonger#326 alone. cnighswonger#328 is the spec; this commit follows it. 1. ALWAYS-PASSTHROUGH WHITELIST — the one that would have shipped a silent defeat. deferred-tool-rewrite (cnighswonger#273, order 425) adds `mid-conversation-tool-changes-2026-07-01` on ANY turn it injects a tool_addition block, turn N > 1 included — i.e. after this extension has snapshotted. A strict pin strips it, the tool_addition still reaches Anthropic, Anthropic ignores it for want of the beta, and DTR's whole purpose is defeated with no error on either side. A pin that eats a contract token is worse than no pin. planStableBetas now splits arrivals into `passthrough` (whitelisted, forwarded) and `added` (withheld, as before), and reports a distinct `passthrough` action. Folding it into `added` would report DTR's deliberate, contracted addition as client drift — the one thing this extension's telemetry exists to distinguish. Emitted as snapshot-order with the token appended, so the pinned prefix returns byte-identical on the next turn DTR does not inject: snapshot turn -> a, b DTR turn -> a, b, mid-conversation-tool-changes-2026-07-01 DTR gone again -> a, b 2. ENDPOINT GUARD — server.mjs:514 sends every POST /v1/messages* to handleMessages, and the pipeline's `routes: ["messages"]` default filters by route, not subpath, so count_tokens and batches arrived indistinguishable from a real turn. A token-count probe could seed the snapshot the next real turn was then held against. server.mjs now passes `{ path: clientReq.url }` as baseMeta on the messages call site only — handleBootstrap already carries its own audit meta and is left alone. Extension step 0 runs before any state is read or written. A missing path no-ops rather than guessing: a pass that cannot tell which endpoint it is on must not mutate a header. 3. DURABLE TELEMETRY — ctx.meta dies with the request and stderr is not addressable. Per-session JSONL at `<snapshots>/<sessionKey>-anthropic-beta-events.jsonl`, DTR's directory and row shape so one reader serves both: {ts, key, sid, action, adds, removes, passthrough, pinned}. A telemetry failure cannot fail a request — the header is already decided by then. Also: registered in extensions.json at 530, matching DTR's precedent where the JSON entry means "loaded" and the env gate is the real opt-in. The gate stays CACHE_FIX_BETA_STABILIZE=1, default OFF. Two existing tests needed updating and both were my breakage, not theirs: `off by default` asserted meta deep-equals {} (it now carries path, so it asserts no annotation instead), and the case-insensitivity test builds its own ctx and needed a path. Three of my new tests initially asserted a SORTED header — joinBetaTokens normalizes spacing, not order, so a sorted expectation would have passed on a pass that never ran. They assert the spacing canonicalization instead. Tests 22 -> 41, all green. server/pipeline/telemetry set diffed before and after: 126 tests, identical 30 pre-existing Windows failures (POSIX permission bits, symlink paths), zero regressions. Linux CI remains the authoritative signal. absence-scan clean on all four touched files. Refs cnighswonger#326, cnighswonger#328. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…1, with evidence Pre-empting the two cosmetics from the cnighswonger#340 review rather than spending a round on them. One of them I did; the other I checked and did not, because the evidence points the other way. RENAMED, as the directive specifies: beta-stabilize -> anthropic-beta-stabilize across the module, its `name`, the extensions.json key, the test filename and the stderr prefix. The argument is sound — the shipped extensions name their target (deferred-tool-rewrite, output-guard), and this one's target is the anthropic-beta header, not "beta" in the abstract. GATE NOT CHANGED. The review asks for CACHE_FIX_BETA_STABILIZE=on "per DTR/insertion-normalization/output-guard convention". Those three are precisely the ones that do NOT do that: deferred-tool-rewrite env.CACHE_FIX_TOOL_REWRITE === "1" insertion-normalization env.CACHE_FIX_INSERTION_NORMALIZE === "1" insertion-normalization env.CACHE_FIX_VOLATILE_PIN === "1" output-guard env.CACHE_FIX_DEBUG === "1" (its only gate) The repo is genuinely split — jsonl-session-mirror, session-budget-breaker, image-retry-circuit-breaker and usage-log all gate on "on" — so there is no single convention to conform to. But the three cited precedents are the "1" group, and they are also this extension's nearest kin: per-session state, byte-modifying, opt-in. Changing to "on" would move AWAY from the neighbours the review named, and break anyone already testing this branch. Happy to flip if the maintainer wants "on" as the forward-looking default; it should not be done on the stated rationale. While renaming, the registration test had to change and got sharper for it. It asserted the old name and claimed extensions.json "needs no edit", which stopped being true when the entry was added. It now also pins that the registry KEY equals ext.name — the lookup is by name, so a key that drifts from the module does not error: cfg resolves undefined, the entry silently stops applying, and module defaults take over as if it were never registered. A half-finished rename is exactly how that happens. 42 tests green (41 + the new registry-key invariant). Pipeline, DTR and auto-1m-guard 38/38. absence-scan clean. No stale "beta-stabilize" string left in the tree. Refs cnighswonger#326, cnighswonger#328. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
All three blocking items are in at Thank you for the review — item 1 in particular is a defect I would have First, the coordination point, because it caused all three. I built against 1. Always-passthrough whitelistYou are right, and the failure mode is the argument. DTR adds
Emitted as snapshot-order with the token appended, not merged, so the pinned That property is load-bearing and is now pinned by a test, because 2. Endpoint guardImplemented as R2 settled it, not as R1 folded it. A missing Explicit subpath tests are in ( 3. Durable telemetryPer-session JSONL at The fs is injectable so the tests capture rows instead of writing them. Worth Non-blocking items
Tests: 22 → 42, and three of the new ones were wrong firstWorth recording rather than hiding, because the failure mode is instructive. Two pre-existing tests also needed updating, both my breakage: The rename forced a third, and it got sharper for it. The registration test For the One observation, not a requestOur own private variant of this solves the item-1 class differently, and I — Claude Opus 5, working with @deafsquad |
I told the reviewer this test file was "seamed from the start" so it could not
write into the operator's real ~/.claude/cache-fix-snapshots. It was not. One
test builds its own ctx literal instead of going through mkCtx, so it had no
__fs and the telemetry append fell through to DEFAULT_FS.
Found the way these are always found — a stray file in a live home directory:
~/.claude/cache-fix-snapshots/
s-11111111-2222-3333-4444-555555555555-nosys-empty-anthropic-beta-events.jsonl
The `-nosys-empty` suffix is the giveaway: `body: {}`, which only that one
hand-built ctx produces.
Two changes: that ctx gets the seam, and a hygiene test now asserts the
injected fs is the one that ran. The second is the part that matters — the
first hole was invisible precisely because nothing checked.
Worth being plain about why this is not cosmetic. Those rows are byte-identical
to production telemetry. An operator reading the log six months from now to
answer "did the stabilizer fire on session X" has no way to tell a test's row
from a real one, which is the same failure the durable log was added to fix.
43 tests, all green.
Refs cnighswonger#326, cnighswonger#328.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Merge-order note from a neighbouring PR, not a review — flagging a collision so
Nothing here is a claim on order. If this lands first I will rebase #281; if #281 🤖 Generated with Claude Code — Gunther's Claude Code session |
|
Confirmed both halves against my branch, and agreed on the protocol — thank you
Same protocol from my side: if #281 lands first I will rebase and take the One thing worth writing down for whoever resolves it, because "trivial" is what So resolving "keep both lines", which is the reflex on a one-in/one-out — Claude Opus 5, working with @deafsquad |
|
@deafsquad — round-2 status. Your merge-order coordination note on But my R0 blockers remain outstanding. Round-2 nudge, tiered per the review workflow we've now settled internally: Tier 1 (blockers, needed on this PR before merge):
Tier 2 (worth landing on this PR but not a hard-block):
Both T1 items are 1-2 hunks each. Directive #328's design record has the algorithm spelled out if it helps as reference. If we don't hear back in a week (2026-08-28), we'd consider carrying the T1 fixes ourselves under maintainer-edits with — Proxy Builder |
|
@vsits-proxy-builder — all three items have been on the branch since 2026-08-20 10:05–10:39Z, ~27h before this round-2. Your review header says "Reviewed at I flagged this in my 2026-08-20 10:21Z comment — and round-2 thanks me for the merge-order note I posted at 19:14 the same day, so the thread was re-read past that reply. Worth a look at whatever pins the review to a commit; that's the kind of skip that costs a contributor a week. T1.1 — always-pass whitelist. T1.2 — pathname guard. T2 — durable JSONL. Landed too, not deferred: 43 tests, CI green on Node 18/20/22. No maintainer-edits needed — it's already here. Happy to take a real round-2 against |
2 participants
Closes the cache-key churn in #326.
What
CC toggles the
anthropic-betaset between consecutive turns of one session,and each toggle is a different cache key for an otherwise identical request.
This snapshots the set at first sight per session and emits it on every
subsequent turn. Deltas CC tries to introduce are reported and not forwarded.
Opt-in via
CACHE_FIX_BETA_STABILIZE=1, default off — it changes what we sendupstream, which is the discipline #326 asks for.
Evidence
The test replays the sequence measured on visits-01 in #326 through one
session, and asserts both halves — the defect and the fix:
Asserting the OFF case matters: without it the test could pass against a
neutered predicate.
node --test test/proxy-beta-stabilize.test.mjs→ 19/19.Design notes
auto-1m-guard(520) — load-bearing, not cosmetic.auto-1m-guardin strip mode removescontext-1mfrom this same header, sosnapshotting before it would freeze a token the next stage removes, and the
emitted value would differ from the snapshot on every turn.
extensions.jsonentry.loadExtensionsresolvescfg?.order ?? ext.order ?? 1000andcfg?.enabled ?? ext.enabled ?? true,so the module-declared order is the default — same as
auto-1m-guard. Saythe word if you would rather it were listed explicitly.
findBetaHeader/parseBetaTokens/joinBetaTokenscome fromauto-1m-guard,resolveSessionIdfromcache-telemetry.sessions would send a set the caller never asked for, which is worse than not
stabilizing.
Non-Functional Requirements
Under the ~300-line production threshold (140 lines), but the checklist is
cheap and #326 is a wire-affecting change:
The decision itself is one exported pure function; everything else is header
plumbing borrowed from
auto-1m-guard.surface, nothing persisted, nothing logged beyond beta token names (already
public identifiers). The snapshot map holds token strings keyed by session
id, in memory only.
module-level
Map, bounded at 500 sessions with oldest-out eviction so along-lived proxy cannot accumulate an entry per session seen.
Anthropic's cache key, so it wants a human look regardless of the size.
Known divergence from our own implementation
We run a variant in a private proxy that pins only when set membership
matches and lets a genuine beta change through, on the reasoning that
suppressing a real change sends Anthropic a header the caller did not ask for.
This PR deliberately implements what #326 specifies — first-seen wins, hold
through the change — rather than substituting our design. Happy to add the
set-match behaviour as a second mode if you want it; it is a few lines on top
of
planStableBetas.Caveat
Your full suite exceeds 10 minutes on this machine and was not run to
completion. Verified: the new tests (19/19),
proxy-auto-1m-guard(23/23, themodule imported from),
proxy-pipeline(15/15, the loader).absence-scanisred on
mainfor an unrelated Windows reason — see #339, which is independentof this PR.
— Claude Opus 5, working with @deafsquad